Automatic picture classifying system and method in a dining environment

ABSTRACT

An automatic classifying system in a dining environment includes a picture uploading component implemented in an electronic device for transmitting a set of pictures via the Internet, and a server for directly or indirectly receiving the set of pictures. The server has a picture analysis component for classifying one of the pictures according to at least two classifications and generating an analysis result to a web-platform system so as to display the picture and the analysis result.

RELATED APPLICATIONS AND PRIORITY CLAIM

This application is a continuation of and claims priority tocommonly-assigned U.S. patent application Ser. No. 15/132,313, filedApr. 19, 2016, and entitled, “Automatic Picture Classifying System andMethod in a Dining Environment,” which claims priority to TaiwaneseApplication No. 104134437 filed Oct. 20, 2015, entitled “AutomaticPicture Classifying System and Method in a Dining Environment,” all ofwhich are herein incorporated by reference in entirety for all purposes.

BACKGROUND 1. Field of the Invention

The present invention relates to a picture classifying system and, moreparticularly, to an automatic picture classifying system in a diningenvironment.

2. Background

Food and beverage (F&B) weblogs on the Internet have become one way ofknowing restaurants in everyday life. However, photos on such blogs areusually not classified or organized, and there exists no classificationmechanism to automatically classify such photos. Moreover, restaurantinformation cannot be gleaned from photos. Therefore, it is desirable toprovide an automatic classifying system with a rapid and automaticclassification to a user and with a more convenient interface to othersfor easily obtaining a restaurant's information, so as to mitigateand/or obviate the aforementioned problems.

SUMMARY

It is an object of the present invention to provide an automaticclassifying system in a dining environment, including a pictureuploading module implemented in an electronic device for transmitting aset of pictures via the Internet, and a server for directly orindirectly receiving the set of pictures, the server including a pictureanalysis module for classifying one of the pictures according to atleast two classifications and generating an analysis result to aweb-platform system so as to display the picture and the analysisresult.

In a preferred embodiment, the picture analysis module includes at leasttwo analyzing modules selected from an appearance analyzing module, astaff analyzing module, a menu analyzing module, a main course analyzingmodule, a drink analyzing module, and a dessert analyzing module.Accordingly, the picture analysis module can classify the picturesaccording to at least two of the appearance classification, staffclassification, menu classification, main course classification, drinkclassification, and dessert classification.

In a preferred embodiment, the server further includes a classificationcheck module to determine whether previous and next pictures of achecked picture among the set of pictures are of the sameclassification. When the previous picture and the next picture are ofthe same classification, the highest classification weight correspondingto the checked picture is compared with a weight approximationcorresponding to the classification of the previous and next pictures,and the checked picture is re-assigned to the classification of theprevious and next pictures when the highest classification weight issmaller than the weight approximation.

Thus, the automatic classifying system can automatically classifypictures selected by a user and publish them in the web-platform system.When other users online access the web-platform system, they can browsethe pictures and rapidly get the desired information via theclassifications. In addition, the automatic classifying system can havea verifying function through the analysis result, so as to have moreaccuracy in the classified result.

It is another object of the present invention to provide an automaticclassifying method in a dining environment, which is performed by anautomatic classifying system. The method includes: using a pictureuploading module to transmit a set of pictures via the Internet; using aserver to directly or indirectly receive the set of pictures; using apicture analysis module of the server to classify one of the picturesaccording to at least two classifications and generate an analysisresult; using the server to transmit the analysis result to aweb-platform system; and using the web-platform system to display thepicture and the analysis result.

In a preferred embodiment, the picture analysis module includes at leasttwo analyzing modules selected from an appearance analyzing module, astaff analyzing module, a menu analyzing module, a main course analyzingmodule, a drink analyzing module, and a dessert analyzing module.Accordingly, the picture analysis module can classify the picturesaccording to at least two of the appearance classification, staffclassification, menu classification, main course classification, drinkclassification, and dessert classification.

In a preferred embodiment, the method further includes using aclassification check module to determine whether previous and nextpictures of a checked picture among the set of pictures are of the sameclassification. When the previous and next pictures are of the sameclassification, the highest classification weight corresponding to thechecked picture is compared with a weight approximation corresponding tothe classification of the previous and next pictures, and the checkedpicture is re-assigned to the classification of the previous and nextpictures when the highest classification weight is smaller than theweight approximation.

Thus, the automatic classifying method can automatically classifypictures selected by a user and publish them in the web-platform system.When other users online access the web-platform system, they can browsethe pictures and rapidly get the desired information via theclassifications. In addition, the automatic classifying method can havea verifying function through the analysis result, so as to allow theclassified result to be more accurate.

In some embodiments, a method is provided for classifying picturesassociated with a dining environment. The method comprises: directly orindirectly receiving, using one or more computing device processors, aset of pictures via Internet from an electronic device; classifying,using the one or more computing device processors, a picture, from theset of pictures, according to at least two classifications; analyzing,using the one or more computing device processors, the picture forobtaining at least two classification weights corresponding to at leasttwo classifications; selecting, using the one or more computing deviceprocessors, a higher classification weight, of the at least twoclassification weights, and a first classification associated with thehigher classification weight; modifying, using the one or more computingdevice processors, the first classification associated with the pictureto a second classification based on a picture-creating time associatedwith the picture; and transmitting, using the one or more computingdevice processors, the picture and the second classification to aweb-platform system.

In some embodiments, modifying the picture is further based on comparinga classification weight of the picture with a weight approximation of aclassification corresponding to a time-checking point.

In some embodiments, the time-checking point is earlier than or laterthan the picture-creating time.

In some embodiments, an automatic classifying method in a diningenvironment, the method comprising: directly or indirectly receiving,using one or more computing device processors, a set of pictures viaInternet from an electronic device; classifying, using the one or morecomputing device processors, a picture, from the set of pictures,according to at least two classifications; analyzing, using the one ormore computing device processors, the picture for obtaining at least twoclassification weights corresponding to at least two classifications;selecting, using the one or more computing device processors, a higherclassification weight, of the at least two classification weights, and afirst classification associated with the higher classification weight;determining, using the one or more computing device processors, a firstpicture before the picture in the set of pictures, and a second pictureafter the picture in the set of pictures, are associated with a secondclassification different from the first classification; re-assigning,using the one or more computing device processors, the picture to thesecond classification; and transmitting, using the one or more computingdevice processors, the picture and the second classification to aweb-platform system.

In some embodiments, re-assigning the picture is further based oncomparing a classification weight associated with the picture with aweight approximation associated with the first picture and the secondpicture.

In some embodiments, an automatic classifying system is provided in adining environment. The system comprises: a server for directly orindirectly receiving a set of pictures, the server including a pictureanalysis component to classify a picture, from the set of pictures,wherein the set of pictures are received from an electronic device,wherein the server includes a picture learning engine for analyzing aplurality of pictures to find a first feature associated with at leastsome pictures from the plurality of pictures, wherein the pictureanalysis component selects a first classification for the picture basedon determining whether the first feature is present in the picture, andwherein the server transmits the analysis result to a web-platformsystem for initiating display of the picture.

In some embodiments, the picture learning engine operates based on adeep learning operation.

In some embodiments, the picture analysis component further selects thefirst classification for the picture based on a first classificationweight associated with the first classification being greater than asecond classification weight associated with a second classification.

In some embodiments, the plurality of pictures are associated with asingle classification.

In some embodiments, the plurality of pictures are associated withmultiple classifications.

In some embodiments, the first feature associated with the at least somepictures from the plurality of pictures comprises a common featureassociated with the at least some pictures.

In some embodiments, the plurality of pictures are analyzed by thepicture learning engine before the server receives the set of pictures.

In some embodiments, the picture comprises at least one of a diningenvironment appearance-related picture, a dining environmentstaff-related picture, a dining environment menu-related picture, a maincourse-related picture, a drink-related picture, or a dessert-relatedpicture, and the picture learning engine operates based on a deeplearning operation.

In some embodiments, another automatic classifying system is provided ina dining environment. The system comprises: a server for directly orindirectly receiving a set of pictures, the server including a pictureanalysis component to classify a picture, from the set of pictures,wherein the set of pictures are received from an electronic device,wherein the picture comprises at least one of a dining environmentappearance-related picture, a dining environment staff-related picture,a dining environment menu-related picture, a main course-relatedpicture, a drink-related picture, or a dessert-related picture, whereinthe picture analysis component selects a first classification for thepicture based on prior classification of one or more pictures, andwherein the server provides access of the picture to a social mediaplatform, the social media platform displaying the picture, the socialmedia platform enabling one or more users of the social media platformto interact with the picture.

In some embodiments, the prior classification of the one or morepictures is performed by a picture learning engine that identifies acommon feature for at least some pictures from the one or more picturesbefore the server receives the set of pictures, and wherein the pictureanalysis component determines that the common feature is present in thepicture.

In some embodiments, the picture analysis component selects the firstclassification for the picture based on a first classification weightassociated with the first classification being greater than a secondclassification weight associated with a second classification.

Other objects, advantages, and novel features of the invention willbecome more apparent from the following detailed description when takenin conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 schematically illustrates a configuration of an automaticclassifying system in a dining environment according to the invention;

FIG. 2 schematically illustrates a preferred configuration of a pictureuploading module according to the invention;

FIG. 3 schematically illustrates a preferred configuration of aweb-platform system according to the invention;

FIG. 4(A) schematically illustrates a preferred configuration of aserver according to the invention;

FIG. 4(B) is a flowchart illustrating an operation of a server accordingto the invention;

FIG. 5(A) is a flowchart illustrating a check operated by aclassification check module according to the invention;

FIG. 5(B) schematically illustrates a practical process of FIG. 5(A)according to the invention;

FIG. 6(A) is a flowchart illustrating another check operated by aclassification check module according to the invention;

FIG. 6(B) schematically illustrates a practical process of FIG. 6(A)according to the invention;

FIG. 7(A) is a flowchart for another operation of a further checkoperated by a classification check module according to the invention;

FIG. 7(B) schematically illustrates a practical process of FIG. 7(A)according to the invention;

FIG. 7(C) schematically illustrates another practical process of FIG.7(A) according to the invention;

FIG. 7(D) is a flowchart of a mistake blocking process for the furthercheck of FIG. 7(A) according to the invention;

FIG. 8(A) is a flowchart of an automatic classifying method performed byan automatic classifying system in a dining environment according to theinvention;

FIG. 8(B) schematically illustrates a practical operation of a pictureuploading module according to the invention; and

FIG. 8(C) schematically illustrates a practical operation of a socialmedia platform according to the invention.

DETAILED DESCRIPTION

FIG. 1 schematically illustrates a configuration of an automaticclassifying system 1 in a dining environment according to the invention.The automatic classifying system 1 includes a picture uploading module20, a web-platform system 30, and a server 40. The picture uploadingmodule 20, the web-platform system 30, and the server 40 are incommunication with each other via the Internet or a wireless network,for example. The web-platform system 30 and the server 40 can be of thesame device, or the web-platform system 30 is configured in the server40 and connected to the picture uploading module 20 via the Internet.Alternatively, the web-platform system 30 can be configured in anotherdevice. The picture uploading module 20 is implemented in an electronicdevice 21 for transmitting a set of pictures out of the electronicdevice 21. For example, the restaurant's pictures photographed by a userare uploaded to the web-platform system 30. The web-platform system 3receives the pictures from the electronic device 21 and transmits themto the sever 40. In addition, the web-platform system 30 can display thepictures and allow multiple users to concurrently browse them via theInternet. The server 40 includes a picture analysis module 41 forautomatically classifying the pictures and generating an analysisresult, along with the classified pictures, to a web-platform system 30thereby displaying the classified pictures. In other embodiments, thepicture analysis module 41 can directly transmit includes a the picturesto the server 40 for further processing, without passing through theweb-platform system 30. Thus, the pictures photographed by a user in adining experience are uploaded to the Internet through the pictureuploading module 20, such that the server 40 can analyze andautomatically classify the pictures, and publish the classified picturesin the web-platform system 30.

The picture uploading module 20 is implemented in the electronic device21. The electronic device 21 is preferably a computing device with amicroprocessor such as a personal computer (PC), and more preferably aportable mobile device such as a smartphone, notebook computer, tabletPC, and the like. In practice, a device with a network communicationcapability can be regarded as the electronic device 21 with the pictureuploading module 20. The picture uploading module 20 is preferably acomputer program implemented in the electronic device 21 for uploadingthe pictures stored in the electronic device 21 to the Internet, forexample to 20 the web-platform system 30. In some embodiments, thepicture uploading module 20 can be a microprocessor, microcontroller, ortransmission interface in the electronic device 21, or driving softwarefor the hardware of the electronic device 21 for driving the electronicdevice 21 to upload the pictures.

With the computer program, the picture uploading module 20 can have adisplay interface 201 and an operation interface 202. FIG. 2schematically illustrates the detail of the picture uploading module 20according to the invention. The display interface 201 of the pictureuploading module 20 displays a set of pictures. Preferably, the picturesare a set of pictures provided in a period of time, such as a pluralityof pictures in a dining experience of a user. The operation interface202 allows the user to select the pictures to be uploaded, andsubsequently the picture uploading module 20 uploads the selectedpictures to the web-platform system 30 via the Internet. In otherembodiments, the picture uploading module 20 can directly upload theselected pictures to the server 40. In addition, it is noted that thedisplay interface 201 and the operation interface 202 can be integratedwith each other if the electronic device 21 has a touch function,thereby allowing the user to touch and select the pictures.

FIG. 3 schematically illustrates a preferred configuration of theweb-platform system 30 according to the invention. The web-platformsystem 30 includes a social media platform 301 and a storage unit 302.The social media platform 301 can be linked to a plurality of electronicdevices 21 for displaying data, such that the users of the electronicdevices 21 can browse the data through their electronic devices 21. Forexample, after one user uploads a set of pictures to the web-platformsystem 30, the other users can browse the pictures on the social mediaplatform 301. The storage unit 302 stores the data of the web-platformsystem 30. Namely, when the web-platform system 30 receives a set ofpictures, it can store the set of pictures in the storage unit 302.Next, the web-platform system 30 can transmit the stored pictures to theserver 40 via the Internet for analysis and receive analysis resultsfrom the server 40, and finally store the analysis results of eachpicture in the storage unit 302. It is noted that the server 40transmits both the pictures and their analysis results to theweb-platform system 30 when the picture uploading module 20 transmitsthe pictures to the server 40 in order to perform an analysis directly.

FIG. 4(A) schematically illustrates a preferred configuration of theserver 40 according to the invention. The server 40 has a pictureanalysis module 401, a picture learning engine 402, and a database 403.The picture analysis module 401, the picture learning engine 402, andthe database 403 are in communication with each other. The pictureanalysis module 401 is preferably comprised of an appearance analyzingmodule 411, a staff analyzing module 412, a menu analyzing module 413, amain course analyzing module 414, a drink analyzing module 415, and adessert analyzing module 416. In practice, the picture analysis module401 includes at least two of the aforementioned modules 411-416, whichare used by the picture analysis module 401 to analyze classificationsof the pictures. Namely, the pictures photographed by a user anduploaded to the server 40 can be classified as appearance, staff, menu,main course, drink, and dessert classifications. In addition, two ormore of the aforementioned modules 411-416 can be merged into onemodule. For example, a main course and a dessert are regarded as thesame classification in certain situation, and in this case the maincourse analyzing module 414 and the dessert analyzing module 416 are ofthe same module, so the pictures associated with the main course and thedessert are classified as the same classification.

The appearance analyzing module 411 analyzes a picture to determinewhether the picture is based on a restaurant's appearance. The staffanalyzing module 412 analyzes a picture to determine whether the pictureis based on people. The menu analyzing module 413 analyzes a picture todetermine whether the picture is based on a menu. The main courseanalyzing module 414 analyzes a picture to determine whether the pictureis based on a main course. The drink analyzing module 415 analyzes apicture to determine whether the picture is based on a drink. Thedessert analyzing module 416 analyzes a picture to determine whether thepicture is based on a dessert. The picture learning engine 402 is anengine architecture which is written according to an algorithm forautomatically finding the regularity in the pictures. Preferably, thepicture leaning engine 402 is made by deep leaning framework algorithmknown as Caffe. Such a framework algorithm allows the picture leaningengine 402 to automatically find the regularity in a great amount ofpictures. For example, a user inputs, in advance, a great amount ofpictures grouped as the same classification to the server 40, and thepicture leaning engine 402 can find the regularity in the pictures andapply it to analyze future pictures to be uploaded by the user orothers. The database 403 stores a plurality of pictures. Preferably, thedatabase 403 pre-stores a great amount of pictures for eachclassification that are pre-input to the database 403, so that thepicture leaning engine 402 can automatically find a feature of theclassification according to the pictures. For example, one hundredpictures are pre-input into the main course classification, and in thiscase the picture leaning engine 402 extracts or finds a feature from theone hundred pictures and so as to classify pictures received in thefuture based on the feature.

Thus, when the server 40 receives a set of pictures from the pictureuploading module 20 or the web-platform system 30, each of the analyzingmodules 411-416 of the picture analysis module 401 analyzes each picturein the set of pictures. For example, the main course analyzing module414 compares the picture with a correlation of the great amount ofpictures in the main course classification to find a weight of thepicture in the main course classification. When the comparison result isgetting more compliant with the correlation, the value of the weight isincreased. After the analyzing modules 411-416 of each classificationhave obtained the weight of the picture, the picture analysis module 401finds a highest classification weight by comparing the weights of theclassifications so as to assign the picture to the classificationcorresponding to the highest classification weight and finally generatean analysis result. Next, the server 40 transmits the picture and theanalysis result to the web-platform system 30.

FIG. 4(B) is a flowchart illustrating a detailed operation of the server40 according to the invention. First, step S41 is executed, in which theserver 40 obtains a great amount of pictures associated with eachclassification and stores them in the database 403. Next, step S42 isexecuted, in which the picture learning engine 402 performs an operationon each classification stored in the database 403 in order to find acorrelation of multiple pictures in each classification for use as aclassification basis of the classification and store it in the database30. Next, step S43 is executed, in which the server 40 receives a set ofpictures from the outside. Next, step S44 is executed, in which each ofthe analyzing modules 411-416 of the picture analysis module 401analyzes each picture of the set of pictures, wherein each of theanalyzing modules 411-416 obtains the classification basis from thedatabase 30 and compares it with each picture so as to give aclassification weight to the picture for each classification. Thus, eachpicture has six classification weights. Next, step S45 is executed, inwhich the picture analysis module 401 analyzes each picture, assigns theclassification corresponding to the highest classification weight in onepicture to the classification of the picture, and generates an analysisresult. Next, step S46 is executed, in which the server 40 transmits theanalysis result of each picture to the web-platform system 30.

In addition, with reference to FIG. 4(A) again, in another preferredembodiment of the present invention, the server 40 may further include aclassification check module 404. The classification check module 404checks whether the analysis result generated by the picture analysismodule 401 has to be corrected or not. The checking is required due tothat, in the analysis process, a classification mistake may occur whenthe weights of two classifications for the picture are very close. Forexample, there are three pictures actually belong to the dessertclassification, but the middle one of the three pictures is classifiedas the main course classification because it has a weight of the dessertclassification slightly smaller than that of the main courseclassification due to its feature is not obvious in certain photographicsituation. Thus, a classification mistake occurs, and a check andcorrection of the classification check module 404 is required in thiscase.

The classification check module 404 presets a weight approximation foreach classification and uses the weight approximation to check whetherto correct the classification of the picture to be checked. FIG. 5(A) isa flowchart illustrating a detailed operation of the classificationcheck module 404 according to the invention. First, step S51 isexecuted, in which the classification check module 404 is based on asequence of creating pictures, i.e., photographing sequence, to select Npictures, where N is a positive integer greater than three. Next, stepS52 is executed, in which the classification check module 404 determineswhether the first and the last ones of the N pictures are of the sameclassification. When the first picture and the last picture are not ofthe same classification, this check is terminated, and a second check isperformed on a next set of N pictures starting with the second picturein the sequence of creating pictures. When the first picture and thelast picture of the N pictures in the second check are of the sameclassification, step S53 is executed, in which the classification checkmodule 404 compares the highest classification weight of each of themiddle pictures (i.e., every picture except the first and the lastpictures) with the weight approximation corresponding to theclassification of the first and the last pictures. When the highestclassification weight of a picture is smaller than the weightapproximation, step S54 is executed, in which the classification of thepicture is re-assigned to that of the weight approximation. Conversely,step S55 is executed to retain the classification of the picture whenthe highest classification weight of the picture is greater than theweight approximation. Subsequently, a next set of N pictures startingwith the second one of the sequence of creating pictures are selected totake another check.

FIG. 5(B) schematically illustrates a practical process of FIG. 5(A)according to the invention. As shown in FIG. 5(B), the first and thelast pictures of three pictures are of the dessert classification, andthe middle one is of the main course classification. The classificationcheck module 404 compares the highest classification weight of themiddle picture with the weight approximation of the dessertclassification. In this case, the highest classification weight of themiddle picture is six, and the weight approximation of the dessertclassification is seven, so that the classification check module 404re-assigns the classification of the middle picture to the dessertclassification.

Furthermore, in certain situations, the classification check module 404predefines a test number to determine the number of pictures to bechecked on classification, such as twenty pictures to be checked onclassification (i.e., taking the check twenty times), so as to keep thestability of the server. However, such a setting has a disadvantage inthat, when there are more than twenty-one pictures uploaded, the lastone cannot be checked. Therefore, in some embodiments, theclassification check module 404 may perform another check.

FIG. 6(A) schematically illustrates another check operated by aclassification check module according to the invention. First, step S61is executed, in which the classification check module 404 checks whetherthe number of pictures uploaded by a user is greater than the testnumber defined by the system and, if yes, step S62 is executed, in whichthe classification check module 404 compares the highest classificationweight of the last uploaded picture with the weight approximation of apreset last classification, which is typically assigned to the dessertclassification because the last dish normally offers a dessert in adining sequence. When the highest classification weight of the lastuploaded picture is smaller then the weight approximation of the presetlast classification, step S63 is executed, in which the classificationof the last uploaded picture is changed to the preset lastclassification. Next, step S64 is executed, in which N pictures countedbackwards from the last uploaded picture are checked as same as stepsS52-S53, where N is a positive integer greater than three.

FIG. 6(B) schematically illustrates a practical process of FIG. 6(A)according to the invention, in which N is set to be three and the testnumber is seven. As shown in FIG. 6(B), the classification check module404 compares the highest classification weight of the eighth picture 61with the weight approximation of the dessert classification, and theclassification of the picture 61 is re-assigned to the dessertclassification because the highest classification weight is smaller thanthe weight approximation. Next, three pictures counted backwards fromthe picture 61 are checked as same as steps S52-S53, and the middle oneis re-assigned to the dessert classification because its highestclassification weight is smaller than the weight approximation of thedessert classification.

In addition, the classification check module 404 of the invention canoperate a further check. FIG. 7(A) is a flowchart illustrating a furthercheck operated by the classification check module 404 according to theinvention. First, step S71 is executed, in which the classificationcheck module 404 obtains a time checking point for a classification. Thetime checking point for the classification is preferably the first-timepresent point thereof during the creating time of all pictures. Forexample, if a dining duration is typically two hours and a dessert isoffered on the table at one and half hours, users can arrange a timechecking point for the dessert classification at one and half hoursafter the first picture is photographed. It is noted that the timechecking point for each classification is arranged randomly by theusers.

Next, step S72 is executed, in which the classification check module 404checks whether there is an abnormally classified picture existed in aplurality of pictures that have the photographing time earlier than thetime checking point for the classification. For example, if the timechecking point for the main course classification is at half hour afterthe first picture is photographed and the time checking point for thedessert classification is at one and half hours after the first pictureis photographed, i.e., the picture creating time of the dessertclassification is later than that of the main course classification, theclassification check module 404 checks whether there is a picture of thedessert classification existed in the pictures photographed before thetime checking point for the main course classification. There is a verylow probability to have the picture of the dessert classificationpresent before the main course classification. However, in case of thepicture of the dessert classification is present before the main courseclassification, step S73 is executed, in which the classification checkmodule 404 checks whether the highest classification weight of thisabnormally classified picture is smaller than the weight approximationof the classification corresponding to the time checking point. Forexample, when the classification check module 404 finds that one pictureof the dessert classification presents before the time checking pointfor the main course classification, it compares the highestclassification weight of the dessert classification and the weightapproximation of the main course classification. When the highestclassification weight of the dessert classification is smaller than theweight approximation of the main course classification, step S74 isexecuted, in which the classification check module 404 changes thepicture from the dessert classification to the main courseclassification.

FIG. 7(B) schematically illustrates a practical process of FIG. 7(A)according to the invention. As shown in FIG. 7(B), a time checking point71 is set for the main course classification with a weight approximationof 7, and there are pictures of the dessert classification existed inthe pictures that are photographed at the time earlier than the timechecking point 71, wherein one abnormally classified picture 72 (dessertclassification) has the highest classification weight of 5. Because thehighest classification weight is smaller than the weigh approximation ofthe main course classification, the classification check module 404changes the classification of the abnormally classified picture 72 fromthe dessert to the main classification. In addition, the otherabnormally classified picture 73 (dessert classification) has thehighest weight of 8. Because the highest weight is greater than theweigh approximation of the main course classification, the dessertclassification of the picture 73 is retained.

In addition, because different users have different use habits, theaccuracy of assigning the time checking point sometimes has a slightdeviation. Therefore, in one embodiment, the check is configured toexecute all steps S71-S74 when the main course classification of aprevious picture exists immediately before the time checking point, eventhe time checking point is not for the main course classification. FIG.7(C) schematically illustrates another practical process of FIG. 7(A)according to the invention. As shown in FIG. 7(C), a time checking point72 is not for the main course classification, but the previous pictureimmediately before the picture corresponding to the time checking point72 is of the main course classification, so that the classificationcheck module 404 still changes the abnormally classified pictures, whichhave a highest classification weight smaller than the weightapproximation of the main course classification, to the main courseclassification.

Furthermore, the check of FIG. 7(A) may be provided with a mistakeblocking process. FIG. 7(D) is a flowchart illustrating a mistakeblocking process for the further check of FIG. 7(A) according to theinvention. As shown in FIG. 7(D), the classification check moduleexecutes step S731 in order to check the classification weights of theabnormally classified pictures in all classifications. Theclassification check module 404 does not change the classification of anabnormally classified picture when the abnormally classified picture inthe other classifications has a classification weight greater than theclassification weight of a classification corresponding to the timechecking point. For example, the classification check module 404 finds apicture of the dessert classification present before the time checkingpoint corresponding to the main course classification, where the picturehas a classification weight of 3 in the main course classification, aclassification weight of 4 in the drink classification, and aclassification weight of 5 in the dessert classification, while theweight approximation of the main course classification is 8. In thiscase, the highest classification weight of the dessert classification ofthe picture is smaller than the weight approximation of the main courseclassification (5<8), but the classification weight (4) of the drinkclassification for the picture is greater than that of the main courseclassification (3), so that the dessert classification of the picture isnot changed, thereby avoiding a decision mistake.

Accordingly, the automatic classifying system 1 of the present inventionis provided with a function of multiple classification verifications soas to gain a more accurate result in classification.

FIG. 8(A) is a flowchart of an automatic classifying method performed byan automatic classifying system in a dining environment according to theinvention. For clear description, a preferred embodiment is given withreference to FIG. 1 and FIG. 8(A) and, in this embodiment, theelectronic device 21 is a smartphone. The picture uploading module 20 isimplemented in an application (APP) of the smartphone. The pictureanalysis module 401 is comprised of the appearance analyzing module 411,the staff analyzing module 412, the menu analyzing module 413, the maincourse analyzing module 414, the drink analyzing module 415, and thedessert analyzing module 416. Thus, the pictures can be classified intosix classifications

First, step S81 is executed, in which a user can use the pictureuploading module 20 to select a plurality of pictures photographed in adining experience for being uploaded to the Internet. FIG. 8(B)schematically illustrates a practical operation of the picture uploadingmodule 20 according to the invention. As shown in FIG. 8, the pictureuploading module 20 displays multiple pictures on the display interface201, and the user can use the operation interface 202 to select thepictures to be uploaded. In addition, the picture uploading module 20can be linked with a camera on the electronic device 21, such that, whenthe user applies the picture uploading module 20 to drive the camera totake pictures, the pictures uploading module 20 can upload the picturesin real-time.

After the pictures are uploaded, step S82 is executed, in which theserver 40 directly or indirectly receives the uploaded pictures. Next,step S83 is executed, in which the picture analysis module 401classifies the pictures and generates an analysis result for eachpicture. The analysis result of each picture can be checked andcorrected by the classification check module 404. Next, step S84 isexecuted, in which the server 40 transmits the analysis results to theweb-platform system 30 via the Internet. Next, step S85 is executed, inwhich the web-platform system 30 displays the pictures on the socialmedia platform 301 according to their classifications. Thus, thenetizens can be linked to the social media platform 301 via the Internetto browse the classified pictures according to the classifications.

FIG. 8(C) schematically illustrates a practical operation of the socialmedia platform 301 according to the invention. The social media platformhas six operation interfaces 801-806 respectively corresponding to sixclassifications, and each classification has its classified pictures.When an operation interface 801-806 corresponding to one of the sixclassifications is selected by a netizen, the social media platform 301displays the classified pictures of this classification. In addition,the social media platform 301 is connected with an editing interface303, which can fetch instructions externally inputted by the user ornetizens and allow the social media platform 301 to generate answers. Ina preferred embodiment, the editing interface 303 includes a messageinstruction to allow the user or netizens to publish their comments onthe classified pictures. The editing interface 303 also includes acorrection instruction to allow users to manually correct theclassification of a picture. In addition, the social media platform alsohas an information forum 304 to display the information provided byusers and the messages left by the netizens. In addition, in a preferredembodiment, the picture uploading module 20 can display the informationof the social media platform 301. For example, the classified picturesof each classification and the corresponding messages can be simplydisplayed by the picture uploading module 20.

As cited, the invention provides an automatic classifying system in adining environment for automatically classifying pictures photographedby users in a dining experience so as to save a great amount ofmanpower. The system further provides the social media platform forcollecting the pictures and allowing others to conveniently use theinformation of the restaurants. In addition, the system further includesa function of classification verification to gain a more accurate resultin classification.

Although the present invention has been explained in relation to itspreferred embodiment, it is to be understood that many other possiblemodifications and variations can be made without departing from thespirit and scope of the invention as hereinafter claimed.

What is claimed is:
 1. An automatic classifying system in a diningenvironment, comprising: a server for directly or indirectly receiving aset of pictures, the server including a picture analysis component toclassify a picture, from the set of pictures, wherein the set ofpictures are received from an electronic device, wherein the serverincludes a picture learning engine for analyzing a plurality of picturesto find a first feature associated with at least some pictures from theplurality of pictures, wherein the picture analysis component selects afirst classification for the picture based on determining whether thefirst feature is present in the picture, wherein the server transmitsthe analysis result to a web-platform system for initiating display ofthe picture, wherein the server provides access of the picture to asocial media platform, the social media platform displaying the picture,the social media platform enabling one or more users of the social mediaplatform to interact with the picture, wherein the first featureassociated with the at least some pictures from the plurality ofpictures comprises a common feature associated with the at least somepictures, and wherein the picture analysis component selects the firstclassification for the picture based on a first classification weightassociated with the first classification being greater than a secondclassification weight associated with a second classification.
 2. Thesystem of claim 1, wherein the picture learning engine operates based ona deep learning operation.
 3. The system of claim 1, wherein the pictureanalysis component further selects the first classification for thepicture based on the first classification weight associated with thefirst classification being greater than the second classification weightassociated with the second classification.
 4. The system of claim 1,wherein the plurality of pictures are associated with a singleclassification.
 5. The system of claim 1, wherein the plurality ofpictures are associated with multiple classifications.
 6. The system ofclaim 1, wherein the first feature associated with the at least somepictures from the plurality of pictures comprises the common featureassociated with the at least some pictures.
 7. The system of claim 6,wherein the plurality of pictures are analyzed by the picture learningengine before the server receives the set of pictures.
 8. The system ofclaim 7, wherein the picture comprises at least one of a diningenvironment appearance-related picture, a dining environmentstaff-related picture, a dining environment menu-related picture, a maincourse-related picture, a drink-related picture, or a dessert-relatedpicture, and wherein the picture learning engine operates based on adeep learning operation.
 9. The system of claim 1, wherein the servermodifies the first classification associated with the picture to thesecond classification or a third classification based onpicture-creating time data associated with the picture.
 10. The systemof claim 1, wherein the server modifies the first classificationassociated with the picture based on comparing the first classificationweight associated with the picture with a weight approximation of aclassification corresponding to a time-checking point.
 11. The system ofclaim 10, wherein the time-checking point is earlier than or later thanthe picture-creating time.
 12. The system of claim 1, wherein the servermodifies the first classification associated with the picture to thesecond classification or a third classification based on a secondpicture received in the set of pictures being classified as the secondclassification or the third classification and based on time dataassociated with the picture or the second picture.
 13. The system ofclaim 1, wherein the server modifies the first classification associatedwith the picture to the second classification or a third classificationbased on a second picture received in the set of pictures beingclassified as the second classification or the third classification. 14.The system of claim 13, wherein the second picture is positioned orordered immediately before or immediately after the picture.
 15. Thesystem of claim 1, wherein the server modifies the first classificationassociated with the picture to the second classification or a thirdclassification based on a second picture received in the set of picturesbeing classified as the second classification or the thirdclassification and based on a third picture received in the set ofpictures being classified as the second classification or the thirdclassification.
 16. The system of claim 15, wherein the second pictureis positioned or ordered immediately before the picture and the thirdpicture is positioned or ordered immediately after the picture.
 17. Thesystem of claim 1, wherein the server reassigns the first classificationassociated with the picture to the second classification or a thirdclassification based on data not associated with the picture.
 18. Thesystem of claim 1, wherein the server modifies the first classificationassociated with the picture to the second classification or a thirdclassification based on data associated with a second picture or a thirdpicture received, positioned, or ordered before, after, orsimultaneously with the picture.
 19. The system of claim 1, wherein theserver modifies the first classification associated with the picture tothe second classification or a third classification based on dataassociated with a second picture and a third picture received,positioned, or ordered before and after the picture, respectively. 20.The system of claim 1, wherein the server modifies the firstclassification associated with the picture to the second classificationor a third classification based on comparing a classification weightassociated with the picture with a weight approximation associated withthe picture and a second picture received, positioned, or orderedbefore, after, or simultaneously with the picture.